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BACKGROUND COMMENTS 


This report documents the work of one component of a larger research effort. The objective of the 
project is to understand what schools of education are doing to prepare teachers to use data in their 
practice. The issue is multifaceted, complex, and systemic. Schools of education do not act alone to 
suddenly introduce courses on data-driven decision making into their curricula because they have a 
whim to do so or because policymakers say that data literacy among educators is important. Schools 
of education must come to realize on their own that building the human capacity to use data among 
their teacher candidates is a response to needs from the field, stimulated in part by policymakers’ 
rhetoric that education must become an evidence-based field. The study, of which this report is a 
part, contains three distinct, but interconnected components that, in combination, provide a 
depiction of the landscape of teacher preparation and data literacy. The components include a 
survey to schools of education, a review of selected syllabi, and an analysis of state licensure 
documents and requirements. This document focuses on the syllabus analysis. 


BACKGROUND INFORMATION 


The survey and analysis of the state credentialing language provides one lens to examine the 
opportunity teachers have to learn about data literacy in their pre-and inservice courses offered in 
schools of education. The course syllabi provide another such lens that can be used to triangulate 
the findings from the survey analysis. Syllabi present in general terms the content and perspective of 
the course that is being taught (Hess & Kelly, 2007). They identify the topics that will be addressed 
by the instructor and the information that will be read by the students. As Hess and Kelly (2007) 
indicate “syllabi are like blueprints: they reveal structure and design, even if they do not fully reflect 
what real-life instruction looks like” (p. 246). The course readings and the topics of the class 
discourse include one level of information about what is in the course. Stronger indications of the 
opportunity that students have to learn the knowledge and skills of data literacy are found in the 
nature of the assignments that students must complete. We examined the focus of the course 
generally and used the framework of data literacy that is operationalized in the survey to identify the 
focus of each lesson. 

It is important to emphasize that this analysis is descriptive, not evaluative. The syllabi were 
examined for the extent to which they included the categories of knowledge and skill that were 
identified in Mandinach & Gummer (2012, 2013). Unlike the study conducted by Greenburg and 
Walsh, 2012) that sought to rate schools of education for the quality of teacher preparation in 
assessment, the syllabi for this study were not rated against a rubric that assigned them a value based 
on the depth of coverage against some metric. 
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PROJECT DESCRIPTION 


DATA SOURCE 

The 80 syllabi were submitted by a subset of schools of education that responded to the survey. The 
submission of syllabi was voluntary and 48 schools of education provided at least one syllabus. Two 
of the syllabi came from a school of education that did not complete the survey, and the syllabi from 
this school of education was not included in this analysis. One of the schools submitted a single 
syllabus from a course that was intended for educational administrators and was not included in the 
analysis. Ten of the schools of education provided multiple syllabi. All of the multiple syllabi from a 
given school of education were examined, and the one that most directly addressed the issue of data 
literacy was included in this analysis. For instance, some schools of education included a course for 
administrators in their syllabi and the target audience precluded the use of the syllabus for this 
analysis. Similarly, if a course was focused on general topics of educational psychology and no focus 
on data could be ascertained, it was not included in this analysis. A total of 44 syllabi were used for 
the initial analysis, which looked holistically at the course content and type. 

Two additional analyses were conducted for this study. One examined the nature of the assignments 
required of students in the course to ascertain the extent to which they supported the development 
of data literacy. Only complete syllabi that included a description of the course objectives, required 
textbooks and readings, lectures or discussion threads and assignments were used for the analysis of 
the assignments. A total of 35 syllabi included sufficient information for that analysis. The second 
additional analysis examined the nature of the content that was addressed during the sessions of the 
course as seen in the topics in a course calendar. The “course week” was the unit of analysis of the 
final way that the in this study — what the teacher or teacher candidate studied during a week in the 
course. If the syllabus did not include a weekly schedule, the syllabus was not included in this 
analysis. A total of 35 syllabi provided such a calendar and were used for this final analysis. 

The schools of education that provided surveys are a subset of those who responded to the survey. 
As indicated in Table 1, they are largely a representative subset in all of the categories by which the 
schools of education were classified in the original survey. The exception is that there were no 
sectarian colleges or universities represented in the syllabus analysis. 

Table 1: Comparison of school type in survey and syllabus analysis. 


School Type 
(select all that 
apply) 

Respondents 
(n = 206) 

Holistic 
Analysis 
(n = 44) 

Assignment 
Analysis 
(n = 35) 

Course 

Week 
(n = 35) 

Public 

67.3% 

72.7% 

71.4% 

71.4% 

n = 140 

n = 32 

n = 25 

n = 25 

Private-for-profit 

2.9% 

2.3% 

2.9% 

2.9% 

n = 6 

n = 1 

n = 1 

n = 1 

Private - not-for- 

28.4% 

25.0% 

25.7% 

25.7% 

profit 

n = 59 

n = 11 

n = 9 

n = 9 

Land grant 

12.5% 

11.4% 

14.3% 

8.6% 

n = 26 

n = 5 

n = 5 

n = 3 

Traditional state 

8.2% 

11.4% 

8.6% 

11.4% 
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teachers college 

n = 17 

n = 5 

n = 3 

d- 

ii 

c 

Sectarian 

2.9% 

n = 6 

0.0% 

n = 0 

0.0% 

n = 0 

0.0% 

n = 0 

Non-sectarian 

3.4% 

n = 7 

2.3% 

n = 1 

2.9% 

n = 1 

2.9% 

n = 1 

Other 

1.5% 

n = 3 

4.5% 

n = 2 

2.9% 

n = 1 

5.7% 

n = 2 


CODING SYSTEM 

The coding system for this analysis is parallel to the structure of the survey and is based on the data 
literacy framework of Mandinach & Gummer (2011, 2013). The type of course was identified and 
included either stand alone courses or those courses into which the data literacy focus was 
integrated. Stand-alone courses were coded as assessment/measurement or data literacy courses or 
as a combination of these based on the extent to which they included topics of data use beyond 
achievement or affective constructs. Integrated courses included pedagogy, methods, educational 
philosophy, educational or instructional psychology, statistics or other. The other category included 
courses with a focus such as the teacher as researcher or inquiry into classroom practice. 

The focus of the course was determined by the combination of the nature of the assignments and 
the topics addressed in the course weeks of instruction. These overall foci included the design and 
implementation of assessments (including both classroom designed and standardized tests), analysis 
of data, monitoring of student performances, testing and measurement issues, grading of students, 
diagnosis of specific disciplinary difficulties of students (especially reading), evaluation of students 
(with a focus on special education), curriculum based measurement (including response to 
intervention - Rtl), and general assessment. A course syllabus might have multiple foci. 

Most of the course syllabi were not fine-grained enough to determine the nature of the data that 
students had the opportunity to analyze. We can assume that if an assignment is to develop and 
implement the assessment for a unit of instruction, then the data that will be available to analyze will 
be student and class-level data, unless otherwise specified. It was more difficult to determine when 
students had the opportunity to analyze school or district level data unless an assignment included 
specific instructions to do so. When possible, the data types were coded as demographic, 
attendance, behavioral, attitudinal, perceptual, student performance or achievement, summative 
assessment at the classroom or state level, benchmark of interim assessment, diagnostic, or 
longitudinal data. The data from this part of the analysis were not considered sufficiently robust to 
include in this report. 

Assignments that emerged from the analysis of the data are shown in Table 2. A total of 36 courses 
had sufficient description of the assignments of the course to assign codes. The extent and 
importance of the assignments varied across the syllabi. For instance, a unit plan and assessment 
could be the major assignment of a course. Or it could be based on an assignment of the design of a 
unit in another course, and the assessment part of it carried less weight in the course described by 
the particular syllabus. Assignments could also address multiple codes. A unit plan and assessment 
might include a requirement for the design or identification of multiple items as well as the design, 
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implementation and analysis of a summative test. Formative assessment might also be part of this 
type of assessment. 


of course assignments 


Assignment 


Lesson plan and assessment 

Unit plan and assessment 

Summative assessment 

Formative assessment 

Rubric design 

Assessment observation 

Standardized assessment description or evaluation 

Analysis or writing of specific assessment items or tasks 

Assessment observation 

Classroom observation 

Case study of individual student or small group of students 

Data analysis 

Grading philosophy 

Review of assessment literature on specific topic 


Table 2. Typ< 


The various course weeks of instruction were coded according to categories that were aggregated 
from the survey using a process similar to that of Hess and Kelly (2007. Only those courses that had 
a sufficiently explicit calendar of classes with included topics (and readings where available) were 
used. If the syllabi including only objectives, or an outline of course coverage, the syllabus was 
discarded for this part of the analysis. Codes were assigned based on an analysis of the terms used to 
describe the topic that was addressed during the course week and topic of the readings that were 
assigned. Table 3 indicates the major categories that were used and the issues that were addressed in 
these categories. 


Table 2. Major categories for “course week” topic coding 


Topic 

Sub-topics 

Assessment/measurement 

Types of assessment 

Measurement issues, 

Purposes and use of assessment 

Data topics 

Data quality, 

Data collection processes, 

Identification of appropriate data to use 

Data systems 

Data warehouses, 

Student information, 

Instructional management, 

Assessment or diagnostic assessment systems 

Data tools 

Student dashboards 

Analytic processes 

Analyzing data from multiple sources, especially 
standardized tests 

Inquiry or collaborative inquiry processes 

How to engage in collaborative inquiry, 
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How to frame questions to which data can be 
applied for instructional purposes, 

Decision-making topics 

Use research and evidence to inform decisions 

Use data to make instructional decisions 

Use data to differentiate instruction 

Engage in collaborative inquiry with other 
educators 

Make presentations grounded in data to students 
or parents 

Make actionable decisions based on the 
examination of data 

Program decision making 

Statistics 

N orm-referenced 

Criterion-referenced 

Validity and reliability 

Assessment interpretation 

1. Mean, median, mode — central tendency 

2. Standard deviation 

3. Normal curve distributions 

4. Central tendency 

5. Discrimination level 

6. Stanines 

7. Percentile rank 

8. Raw vs. scaled score 

9. Growth index 

Assessment reporting 

Differences in grain size (cohorts, courses, 
grades) 

Reporting levels (scaled scores, percentiles, 
performance levels) 


FINDINGS 

Focus of Courses 

A holistic analysis of the focus of the courses is based on the coding of the title of the course and 
the overall topics of the course assignments and activities. Of the 46 course syllabi that had 
sufficient information for analysis, 35 (76%) were coded as having a major focus on the design, 
implementation and analysis of types of assessments. The texts for these courses typically address 
the knowledge and practices that classroom teachers need to know in order to identify or design 
items to be used in classroom quizzes and tests and tasks that could be used in performance 
assessments. A secondary focus of these courses was on formative assessment, state assessments or 
assessment policy issues. The courses ranged in terms of the rigor with which assessment was 
addressed. Three courses with a focus on educational measurement included a significant focus on 
statistics, while the majority of the courses that addressed little if any statistics and focused on 
assessment more globally. The syllabi represented courses that are taught at both the undergraduate 
and graduate levels. Given the undergraduate status of the course and the morning or early 
afternoon time at which they were taught, 23 (50%) of the courses were designed for preservice 
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teachers. It was not clear whether the other courses were specifically designed for inservice teachers 
or whether they represent a mix of preservice and inservice contexts. 

Only one course had a specific focus on data-driven decision making in the title, included texts that 
were focused on the development of data literacy as well as assessment texts, and included a specific 
focus on the use of data beyond just achievement or affective data. This course focused on 
classroom management. Three of the assessment courses had a stronger focus on data literacy. One 
was a course that integrated assessment and a broader focus on data associated with class sessions 
that were held in a school. That course will be discussed in more detail below. Two special education 
courses that were focused on assessment of exceptional students also included a deeper focus on the 
use of data. 

The remaining 10 courses addressed a range of foci including general educational psychology, 
learning in the content area, methods, teaching practica, and teacher as researcher. The focus on data 
literacy in these courses was much more limited as will be described in more detail later in this 
report. These courses were included in the analysis as all had at least one week of the course focused 
on assessment or one assignment that had students collecting and analyzing data from individual 
students, small groups, or whole classrooms. 


Assignmen ts 

The coding of the nature of the assignments focused on what was explicitly required of the students. 
These assignments ranged from short term weekly journal entries, reflections and quizzes to long 
term projects, papers, capstone projects, portfolios, field-based assignments connected to projects, 
and assessments and final exams. A total of 36 syllabi included adequate descriptions of the 
assignments 

Lesson or unit plan with assessments: Of the 132 separate assignments analyzed in this study, 
one of the two most frequently identified assignments was some sort of lesson or unit plan with 
assessments. Twenty three of the 36 courses (66%) had either a single lesson or a full unit plan for 
which an assessment task or tasks was identified or designed, implemented and analyzed. Eight of 
these courses (23%) required the student to choose only one lesson, which might be the focus of an 
observation in a cooperating teacher’s classroom. Others were full unit plans with assessments and 
item analysis or design, identified in 15 or (43%) of the syllabi. Unit plans frequently included a 
requirement that the data from the assessments be analyzed with a focus on how the results 
informed instruction. However, it was not possible to determine whether that information was to 
have been used during the course of instruction with immediate consequences for students’ 
opportunities to learn or to plan for future use of the unit plan to teach the unit in the subsequent 
year. Most of the unit plans addressed a range of assessments from formative assessments that are 
meant to be used in teaching, to quizzes and unit tests. In two cases, the assessments were added 
onto a unit plan that was the assignment in another course. The 15 courses that had the analysis or 
design of assessment items typically also had the student practice writing a range of assessment 
types, including selected and open response items as well as portfolio design. 

Analysis or writing of assessment items: Almost half of the courses, 16 (44%) included an 
assignment that had students designing assessment items and tasks. On several occasions, the 
students were required to analyze items that they found in text books or that were used in their 
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teaching placements. Students practiced a range of item types and formats, and some of the courses 
spent a majority of the course time with a focus on item design. 

Summative assessment: An assignment that was focused on the design and use of a summative 
assessment was included in 16 (44%) of the syllabi. These assignments included both quizzes that 
the students designed for use in their classrooms and tests that were included with the assessment 
system for the plan of a unit of instruction. 

Analysis of data: Fourteen (39%) of the courses had an assignment with a specific focus on the 
analysis of data. These assignments were typically based on the classroom summative assessments 
that were part of the lesson or unit analysis. They also included assignments where the student 
teaching candidate collected data from individual students to whom a battery of assessments had 
been given. The focus of the data analysis was to either provide a description of student learning or 
to articulate some future teaching that might be needed based on the performance of the children. 
Five of the courses had explicit language around the use of the data from assessments for 
instructional decision making. Three of these course assignments indicated that the data analysis 
would be conducted in class and focus on the use of state test data, but the specifics of how this 
would happen was not clear. 

Rubric design: Almost a third of the courses included an assignment that had the students 
developing or using a rubric to evaluate student performances. In 11(31%) of the courses, the rubric 
design was specified either as part of the unit assessment system or as a stand-alone assignment. 
These rubrics were most frequently associated with the design and implementation of a particular 
performance task such as a writing assignment or a laboratory report. 

Formative assessment: Relatively fewer, ten (28%) of the courses, had an assignment that was 
focused specifically on formative assessment. These assignments ranged from the design and 
implementation of a formative assessment with a class or a group of students to the analysis of a 
reading focused on formative assessment practices. 

Statistical analyses: The ways in which some form of statistics was addressed in the courses varied 
considerable. Only nine (25%) of the courses had assignments that focused on statistical analyses. 
These might be based on data that was provided in class or that the students needed to obtain from 
their school placements. The nature of the data that were the focus of these assignments was 
difficult to ascertain from the description of the assignment in the syllabi. 

Case studies: Seven (19%) of the courses had an assignment that was focused on the case study of 
a student or a small group of students. These assignments occurred in courses that were intended 
for special education or reading teachers. 

Portfolio assessment: Seven (19%) of the courses included an assignment that addressed portfolio 
assessment in some form. In four of these courses, students were required to reflect on an article 
about portfolio assessment rather than design and implement elements of such a portfolio. 
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Table 3 presents a summary of these findings of the major assignment types. 


Table 3: Distribution of major assignment types 


Assignment 

Lesson 
or unit 
plan 

Assessment 

items 

Summative 

assessment 

Analysis 
of data 

Rubric 

design 

Formative 

assessment 

Statistical 

analysis 

Case 

studies 

Portfolio 

Number 

23 

16 

16 

14 

11 

10 

9 

7 

7 

Percent 

66% 

44% 

44% 

39% 

31% 

28% 

25% 

19% 

19% 


Table 4 indicates the frequency of assignments that occurred in less than 15% percent of the 
courses. These assignments included grading plans or philosophies, observational assessments, or 
literature reviews. 


Table 4: Remaining assignment types 



Assignment 


Grading 
plan/ or 
philosophy 

Observation 

Literature 

Review 

Number of 

courses 

3 

5 

2 

Percent of 

courses 

8% 

14% 

6% 


The analysis of assignments provides additional support for the findings based on the holistic focus 
of the course. As was the case in the initial analysis, the assignments of the courses are largely those 
that address assessment design and implementation with analysis a distant second. The inclusion of 
an assignment on data analysis was identified in only 14 of the courses. The assignment to analyze 
data was typically a general request to consider the implications of the findings from the designed 
and implemented assessments on teaching, rather than a specific focus on what the analysis meant 
for student learning. 

The focus of the assessment development, implementation and analysis was typically at the level of a 
particular subject matter or grade level class or unit within a course. Given the number of course 
syllabi that came from courses for preservice teachers, this focus is understandable as the intent of 
these courses is to prepare students to engage in their practicum of student teaching. Overall, the 
assignments included in the syllabi do not show a significant focus on building data literacy apart 
from assessment. 

It is telling that none of the assignments appeared to have the students introduced to or use any 
particular data system or tool. In only one course were such tools the focus of classroom work of 
discussions as will be described in the next section. The only technologies integrated into these 
courses were instructional management systems, such as Blackboard, and teacher portfolio systems, 
such as Chalk and Wire. In only one course were the students required to use spreadsheets such as 
Excel. 

Course Week Analysis 

A total of 35 syllabi had sufficient information about the topics that were addressed in the course 
identified with a particular “course week” to support the analysis of the categories of topic content. 
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The courses analyzed included a range of weeks of the course, from 6 weeks for a practicum course 
to 16 weeks for some of the semester level courses. Several of the courses met more than once a 
week, and the topic addressed across the multiple sessions in the week as assigned to only one 
category. A total of 419 course sessions were coded. 

Assessment and measurement: This was the category most frequently identified in the course 
sessions, assigned to 270 (64%) of the course weeks. This category included the purposes and uses 
of assessments, types of assessment and multiple other assessment and measurement topics. Some 
of the assessment courses had all by the introductory session and the final session focused on one or 
more types of assessment items or tasks. Other assessment and measurement topics included 
introductions to state and federal assessment, such as the PARC and Smarter Balanced assessment 
systems assuming that their states are participating in one or the other. The representation and 
communication of assessment information was the focus of another 5 of the course weeks. 
Measurement topics included concepts such as reliability and validity, as well as the issues of 
differences in reporting levels from the perspectives of cohorts, courses and grades. 

Other: The second most frequent category of the topic of the course week was the “other” category 
which included discussions and classroom activities that included lesson planning, standards, 
disciplinary content discussions, learning of students, behavior, motivation, educational philosophy 
or additional pedagogical topics. A total of 63 (15%) of the sessions were coded into the “other” 
category. Given the number of courses that were included in this analysis that were provided by the 
schools of education to show how data literacy was incorporated into multiple courses, this 
preponderance of other topics as the focus of the course week makes sense. 

Analytic processes: Analytic processes were difficult to identify in the sessions that occurred over 
time in a course. A total of 25 (6%) course weeks appeared to address these processes. Topics that 
addressed analysis ranged from how to organize data, discuss data with other colleagues or students 
or parents, and how to identify data sources other than achievement. It also included how to use 
data from assessments in order to influence teaching, which was the most prevalent instance of this 
topic. It is interesting that these topics were as frequently the focus of the course week as they were, 
and this emphasizes the extent to which the courses identified in this study emphasized assessment. 

Statistics: Sixteen (4%) of the sessions focused specifically on statistics. Most of the courses 
included only a cursory coverage of statistics, introducing the students to measures of central 
tendency. Several of the courses, especially the measurement courses included multiple sessions on 
statistics, introducing students to different kinds of analysis and considering statistical error in 
measurement. 

Data topics: Eleven course sessions (3%) were identified to address data topics. These topics 
included data quality, the identification of appropriate data to use for particular decisions, and data 
collection methods. The final code, data collection methods, differed from the design and 
implementation of assessment items and tasks, and was found in only 3 of the courses that were 
designed for special education teachers. 

Data-driven decision making: The course weeks were rarely labeled as addressing data-driven 
decision making specifically. These were largely identified by the nature of the reading that was 
assigned as the reading for the course session and were identified in only 7 (2%) of the course 
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weeks. These topics included the use research and evidence to inform decision, the use data to make 
instructional decisions, the use of data to differentiate instruction, and program decision making 

The specific category of inquiry around data was the focus of only four (<1%) class sessions. Two 
of the courses that were focused on special education included group work with students were 
required to work in collaborative groups to identify problems of specific student performance 
around which they were to design their interventions. In a third class, the students were assigned to 
groups in order to design and implement their assessment tasks. 

The data system category was assigned to only four (<1%) class sessions and the discussion of 
data tools was included in only one class session. These class sessions focused on readings that 
included a brief discussion of data systems and tools. This lack of emphasis on data systems and 
tools again reflects the focus of the courses on assessment literacy rather than data literacy. In only 
one course were the students provided with the opportunity for exploring the nature and use of data 
systems, and that was a course that included half of the class sessions in a local high school. In this 
course, the objectives of two course sessions included the following: 

• To identify the basic structure and philosophy of the DDM systems 

• To implement common and state-of-the-art technologies used by schools to collect, analyze 
and report on student outcomes 

• To understand how to use technological systems and information technologies 

• To collect, retrieve and analyze student-learning outcomes 

• To realize the structure and function of relational databases 

This course included assignments that required that students develop a unit plan with assessments, 
implement a formative assessment with a group of students, and analyze and write assessment items. 
It is one of only three courses that required students to write a grading plan or philosophy. 

Two of the codes included in these final two categories were difficult to determine from the syllabi 
and may be underrepresented in the findings. These include data topics of data collection 
processes, which may logically have been integrated within the discussion of the design of different 
assessment items and tasks and identification of appropriate data to use, which may have been 
incorporated in the instruction of which assessment tasks or items to use for which purposes. From 
a data literacy perspective, these two codes are used in an assessment rather than a data literacy 
perspective as they focused on the design and implementation of assessments for classroom or 
individual student use with an apparent focus on individual and student achievement or affect. 

The category of data decision making included two additional codes that were difficult to see in the 
topics assigned during the course weeks. It was difficult to see how the courses specifically 
addressed using data to make instructional decisions and using data to differentiate 
instruction. Again, these activities might have been included in the discussion of assessment use, or 
in instructional design, but they were not specifically associated with data use. 

Table 5 provides the overall distribution of topics by course weeks. Overall, the analysis of the 
content addressed during the course weeks affirms the findings of the holistic examination of course 
focus and the analysis based on the assignments. These courses provided by the schools of 
education in response to a request for experiences students had with data literacy largely address 
assessment literacy. 
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Table 5:1 

Arcentage of course weeks that addressee 

data literacy topics 

Topic 

Assessment 

and 

Measurement 

Assessment 

Representation 

Analytic 

Processes 

Statistics 

Data 

Topics 

Data- 

driven 

Decision 

Making 

Data 

Systems 

Data 

Tools 

Number 

270 

4 

25 

16 

n 

7 

4 

4 

Percent 

64% 

<1% 

6% 

4% 

3% 

2% 

<1% 

<1% 

A total o 

7 63 course wee 

es were assigned to the “other” category 


CAVEATS, CONCLUSIONS AND NEXT STEPS 


There are limitations to this part of the study that need careful consideration. The courses provided 
were volunteered by the schools of education, relying on the knowledge of the programs by the 
person(s) responding to the surveys. There is the possibility that the request for the course was 
relayed from person to person until someone responded with a course that he or she felt would fit 
the request. For this analysis, only one course per institution was used, typically the course that 
appeared to most strongly address the use of data. It is possible that the course chosen for the 
syllabus analysis is not the course that was at the center of the respondent’s consideration when he 
or she filled out the survey. It is possible that the analysis of a suite of courses would provide a 
better picture of how a focus of data across a number of courses was developed, but the initial 
analysis of the multiple course sets did not support such a hypothesis. The sample of schools of 
education who provided multiple courses is quite small. 

Syllabus analysis provides only an abbreviated record of what a particular course provides as an 
opportunity to learn to students. David Labaree is quoted in Keller (2003, p.8) as describing syllabi 
as nothing more than “an ideological portrait” that provide little substantive information about what 
a course addresses. However, we agree with Hess and Kelly (2007, p.246) that “syllabi are like 
blueprints: they reveal structure and design, even if they do not fully reflect what real-life instruction 
looks like”. The syllabus analysis is only one element of this study, and it is based on a limited 
sample of volunteered syllabi. However, it provides one mechanism to triangulate the data from the 
survey and from the analysis of certification regulations to begin to look inside the survey responses 
to develop a better picture of how schools of education operationalize what they identify as data 
literacy. 

The findings across the three types of analysis, holistic focus of the course, analysis of the student 
assignments and identification of the course content through analysis of the course weeks provides a 
picture of how the schools of education appear to be focused on assessment rather than on the 
broader picture of data literacy that has emerged as a crucial function of what teachers should know 
and be able to do in the school. The holistic analysis indicates that the majority of the courses are 
focused on assessment. The assignments required of students predominantly involve the design and 
review of assessment items or the assessment plans for lessons or units of study. The number of 
course weeks assigned to the discussion of assessment and measurement design issues is the 
majority of the time spent in these courses. 

The preponderance of courses that focused on assessment design and implementation brings into 
question the ways in which the school of education faculty and administrators who supplied the 
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course syllabi are interpreting the focus of the course on data literacy. The request for the course 
syllabus in the survey came before the questions that asked about specific data literacy knowledge 
and skills that made up subsequent survey items. However, it is also possible that the survey 
respondents were using the specific sets of knowledge and skills that are listed in the survey to 
construct the map onto the courses taught in their institutions that they felt best aligned with the 
request. 

The survey analysis provides an intriguing glimpse into the opportunity preparing teachers have to 
develop data literacy. A frustrating limitation of the syllabi prompted us to frequently wish we had 
the opportunity to talk to the faculty member, to ask for additional details about some of the 
assignments or in-class experiences to determine whether or not the students were focused on more 
than just assessment data and experiences. It was also impossible to determine the ways in which a 
focus of data was seen as a thread that connected the courses within a program when the schools of 
education volunteered more than one course. The next logical step of this study would be to 
conduct case studies of a selection of schools of education to have the opportunity to examine in 
more detail the assumptions of intent that are evident in the coding of the syllabi. Such case studies 
might include the assessment course that had the students working in the schools and experiencing 
the use of data technology. It would also include a sample of the schools whose courses for special 
education students appeared to have a stronger focus on data literacy than on assessment literacy 
with the opportunity to examine the reasons why such might be the case. A set of case studies might 
also examine how and why the respondents provided assessment and measurement courses in 
response to a survey on data literacy to get a better understanding of their perceptions of data 
literacy. 
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